Self-initialized Visual Servoing for Accurate End-effector Positioning

Vision-based control of an end-effector is important for industrial and domestic robotic applications. Direct positioning based on object recognition or pose estimation output is challenging due to inaccuracies of the estimations and system calibration. Visual servoing overcomes these inaccuracies, however, present techniques require initialization of a target region in the input image. This work presents an integrated system to servo towards a target with automatic initialization. In our two-step approach, the first phase positions the end-effector relative to an object using the output of a pose estimation network. The initial coarse alignment is then refined in a second phase using image-based visual servoing (IBVS). The target region in the image for IBVS is initialized by the detection of the pose estimator. The system monitors the tracking quality of the IBVS to automatically switch from the first to the second phase. Experiments with a 7 degree of freedom robotic arm show that our approach results in successful end-effector positioning from arbitrary initial positions.